A Model for Learning Systems

A model f o r l e a r n i n g sys tems i s p r e s e n t e d , and r e p r e s e n t a t i v e A I , p a t t e r n r e c o g n i t i o n , and c o n t r o l sys tems a r e d i s c u s s e d i n t e rms o f i t s f r a m e w o r k . The model d e t a i l s t h e f u n c t i o n a l components f e l t t o b e e s s e n t i a l f o r any l e a r n i n g s y s t e m , independent o f t h e t e c h n i q u e s used f o r i t s c o n s t r u c t i o n , and t h e s p e c i f i c env i r onmen t i n wh i ch i t o p e r a t e s . These components a re P r f o r m a n c e e l e m e n t , i n s t a n c e s e l e c t o r , c r i t i c , e a r n i n g e l e m e n t , b l a c k b o a r d , and w o r l d m o d e l . C o n s i d e r a t i o n o f l e a r n i n g system d e s i g n l e a d s n a t u r a l l y t o t h e concep t o f a l a y e r e d s y s t e m , each l a y e r o p e r a t i n g a t a d i f f e r e n t l e v e l o f a b s t r a c t i o n . D e s c r i p t i v e Terms: a d a p t a t i o n , l e a r n i n g , c o n c e p t f o r m a t I o n , i n d u c t i o n , pe r fo rmance e l e m e n t , i n s t a n c e s e l e c t o r , c r i t i c , l e a r n i n g e l e m e n t , b l a c k b o a r d , w o r l d m o d e l , m u l t i l a y e r e d sys tems . 1 I n t r o d u c t i o n L e a r n i n g sys tems have been t h e s u b j e c t o f w ide r e s e a r c h i n t e r e s t f o r a number o f y e a r s . The te rms a d a p t a t i o n , l e a r n i n g , c o n c e p t f o r m a t i o n , i n d u c t i o n , s e l f o r g a n i z a t i o n , and s e l f r e p a i r have a l l been used i n t h e c o n t e x t o f t h i s s t u d y . L e a r n i n g system (LS) r e s e a r c h has been conduc ted w i t h i n many d i f f e r e n t s c i e n t i f i c c o m m u n i t i e s , however , and t h e s e t e rms have come to have a v a r i e t y o f mean ings* I t i s t h e r e f o r e o f t e n d i f f i c u l t f o r members o f t h e s e commun i t i es t o r e c o g n i z e t h a t p rob lems w h i c h appear u n r e l a t e d a s a r e s u l t o f v a r i a t i o n s i n t e r m i n o l o g y may i n f a c t be i d e n t i c a l . L e a r n i n g system models as w e l l a r e o f t e n t uned t o t h e r e q u i r e m e n t s o f a p a r t i c u l a r d i s c i p l i n e and a r e not s u i t a b l e f o r a p p l i c a t i o n i n r e l a t e d d i s c i p l i n e s . We have t h e r e f o r e s y n t h e s i z e d a new LS model w h i c h p r o v i d e s a common language f o r u n i f i e d c h a r a c t e r i z a t i o n o f sys tems c o n s t r u c t e d f rom a number o f d i f f e r e n t p e r s p e c t i v e s . T h i s model encourages e x a m i n a t i o n o f t h e s t r e n g t h s and weaknesses o f t h e i n d i v i d u a l f u n c t i o n a l components necessa ry f o r any l e a r n i n g s y s t e m . Because t h e model enab les a d e s i g n e r to i s o l a t e t h e s e f u n c t i o n a l components and s p e c i f y t h e i n f o r m a t i o n wh i ch must be a v a i l a b l e t o them", i t i s p a r t i c u l a r l y u s e f u l a s a parad igm f o r new l e a r n i n g s y s t e m s . I n t h e c o n t e x t o f t h i s p a p e r , a l e a r n i n g system i s c o n s i d e r e d to be any system wh ich uses i n f o r m a t i o n o b t a i n e d d u r i n g one i n t e r a c t i o n w i t h i t s e n v i r o n m e n t t o improve i t s pe r fo rmance d u r i n g f u t u r e i n t e r a c t i o n s . T h i s d e f i n i t i o n i s i n t e n t i o n a l l y b road and may i n c l u d e man/machine systems i n wh i ch humans t a k e on a c t i v e r o l e s as r e q u i r e d f u n c t i o n a l components• 11 T h i s work was s u p p o r t e d by t h e Advanced Research P r o j e c t s Agency under c o n t r a c t DAHC 1573-C-0UJ5 . t h e N a t i o n a l I n s t i t u t e s o f H e a l t h under g r a n t RR 00612-07 . t h e Nava l A i r Systems Command under c o n t r a c t N0019-76 -C-0250 , and t h e N a t i o n a l Sc ience F o u n d a t i o n under c o n t r a c t GK-M1972. Reid Smi th i s s u p p o r t e d by t h e Research and Development Branch o f t h e Depar tment o f N a t i o n a l Defence o f Canada. C . R i c h a r d Johnson , J r . p r o v i d e d v e r y h e l p f u l comments on a d a p t i v e c o n t r o l s y s t e m s . We a l s o r e c e i v e d many v a l u a b l e s u g g e s t i o n s f r om members o f t h e H e u r i s t i c Programming P r o j e c t a t S t a n f o r d * I n t h e f o l l o w i n g s e c t i o n s w e w i l l summarize two d i f f e r e n t approaches t o t h e c o n s t r u c t i o n o f sys tems t h a t can b e s a i d t o l e a r n . The f i r s t app roach c e n t e r s on t h e concep t o f an a d a p t i v e system and i s p r i m a r i l y a s s o c i a t e d w i t h r e s e a r c h i n p a t t e r n r e c o g n i t i o n and c o n t r o l t h e o r y : t h e second i s t h a t o f a r t i f i c i a l i n t e l l i g e n c e ( A l ) . 2 A d a p t i v e System Approach to L e a r n i n g I n t h e c o n t r o l l i t e r a t u r e , l e a r n i n g i s g e n e r a l l y assumed to be synonymous w i t h a d a p t a t i o n , and i s o f t e n v iewed a s e s t i m a t i o n o r s u c c e s s i v e approx imat i o n o f t h e unknown p a r a m e t e r s o f a m a t h e m a t i c a l s t r u c t u r e w h i c h i s chosen by t h e LS d e s i g n e r to represent t h e system under s t u d y [ 6 ] [ 1 0 ] . Once t h i s has been done , c o n t r o l t e c h n i q u e s known t o b e s u i t a b l e f o r t h e p a r t i c u l a r chosen s t r u c t u r e can be a p p l i e d . Thus t h e emphasis has been on pa rame te r l e a r n i n g , and t h e ach ievement o f s t a b l e , r e l i a b l e pe r f o rmance [ 2 5 ] . Prob lems a r e commonly f o r m u l a t e d i n s t o c h a s t i c t e r m s , and t h e use o f s t a t i s t i c a l p r o c e d u r e s t o a c h i e v e o p t i m a l pe r fo rmance w i t h r e s p e c t t o some pe r f o rmance c r i t e r i o n such a s t h e p r o b a b i l i t y o f c o r r e c t p a t t e r n c l a s s i f i c a t i o n , o r mean square e r r o r , i s s t a n d a r d [ 3 3 ] . There a r e many o v e r l a p p i n g and somet imes c o n t r a d i c t o r y d e f i n i t i o n s o f t h e t e rms r e l a t e d t o a d a p t i v e s y s t e m s . The f o l l o w i n g s e t , f o r m u l a t e d b y G l o r i o s o 111 ] s e r v e s t o i l l u s t r a t e t h e main f e a t u r e s . An a d a p t i v e sys tem is d e f i n e d as a sys tem wh i ch responds a c c e p t a b l y w i t h r e s p e c t t o some pe r f o rmance c r i t e r i o n i n t h e f a c e o f changes i n t h e e n v i r o n m e n t o r i t s own i n t e r n a l s t r u c t u r e , A l e a r n i n g sys tem i s a n a d a p t i v e sys tem t h a t responds a c c e p t a b l y w i t h i n some t i m e i n t e r v a l f o l l o w i n g a change i n i t s e n v i r o n m e n t , and a s e l f r e p a i r i n g sys tem i s one t h a t responds a c c e p t a b l y w i t h i n some t i m e i n t e r v a l f o l l o w i n g a change i n i t s i n t e r n a l s t r u c t u r e . F i n a l l y , a s e l f o r g a n i z i n g sys tem i s a n a d a p t i v e o r l e a r n i n g s y s t e m i n w h i c h t h e i n i t i a l s t a t e i s unknown, random, o r u n i m p o r t a n t . Other t e rms o f t e n used t o d e s c r i b e l e a r n i n g sys tems i n t h e p a t t e r n r e c o g n i t i o n and c o n t r o l l i t e r a t u r e a re " s u p e r v i s e d " and " u n s u p e r v i s e d " l e a r n i n g [ 5 l [ 1 0 J . S u p e r v i s e d l e a r n i n g , o r " l e a r n i n g w i t h t e a c h e r " , assumes t h e e x i s t e n c e o f an e x t e r n a l e n t i t y ( u s u a l l y a human) w h i c h p r e s e n t s t h e sys tem w i t h a s e t o f t r a i n i n g i n s t a n c e s , e v a l u a t e s t h e pe r fo rmance o f t h e sys tem f o r t h o s e i n s t a n c e s , and p r o v i d e s t h e c o r r e c t r e s p o n s e s . U n s u p e r v i s e d l e a r n i n g , o r " l e a r n i n g w i t h o u t t e a c h e r " , assumes t h a t t h e e n v i r o n m e n t p r o v i d e s a l l i n s t a n c e s , bu t does n o t p r o v i d e t h e c o r r e c t r e s p o n s e s . Per fo rmance i s t o b e e v a l u a t e d b y t h e sys tem i t s e l f . T s y p k i n [ 2 8 ] has p o i n t e d o u t t h a t u n s u p e r v i s e d l e a r n i n g i s somewhat o f a n i l l u s i o n i n t h e sense t h a t a t e a c h e r / d e s i g n e r d e f i n e s t h e s t a n d a r d s wh i ch d e t e r m i n e t h e q u a l i t y o f o p e r a t i o n o f t h e L S a t t h e o u t s e t , w h e t h e r o r n o t h e i s p r e s e n t d u r i n g t h e a c t u a l o p e r a t i o n o f